Collection Analysis and Use of Viewer Behavior

ABSTRACT

Aspects discussed herein relate to systems, apparatuses, and methods to determine when a user remains engaged or stops paying attention to a content item. In response, a snapshot of the user&#39;s activity and environment may be obtained to ascertain the reason the user stopped paying attention. If the user stopped paying attention as a result of being uninterested in the content item, the system may replace the content item, such as an advertisement, with a different content item to regain the user&#39;s attention. Further, demographic information and the user&#39;s interests may be updated for use in scheduling future content items.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/019,396, titled “Collection Analysis and Use of Viewer Behavior,filed Feb. 9, 2016, which is hereby incorporated by reference in itsentirety.

BACKGROUND

Many entities advertise goods and/or services using commercials ontelevision, webpages, and/or via other media. Advertisers may prefer totarget advertisements based on whether a consumer is likely to beinterested in their goods and/or services. However, conventional methodsof determining targeting information can be unreliable and inaccurate.There is an ever-present need for more reliable and accurate ways totarget advertisements and other content to specific consumers andconsumer demographic groups.

SUMMARY

Various features described herein may provide targeting information foradvertisements and other content. Various features may further providetechniques for re-engaging a user if the user stops viewing or otherwiseconsuming content. For example, content management functionalities maybe implemented to maintain a user's interest.

According to some aspects, a consumption monitoring system may monitor auser to determine the user's attentiveness to a content item beingpresented by a content presentation device such as a television, acomputer, a smart phone, a tablet, a radio, etc. In response to adetermination that the user is not attentively focusing on the contentitem, the consumption monitoring system may determine and/or otherwiseobtain information about (e.g., take a snapshot of) the user's activityand environment. The consumption monitoring system may, based on suchinformation, determine the user's interest level in the content item.For example, the consumption monitoring system may determine, based onthe information, whether there is an extrinsic environmentalcircumstance or event that may have caused the user to stop attentivelyfocusing on the content item. For example, from the information, theconsumption monitoring system may determine whether a doorbell, homephone, or mobile phone rang, or whether another event occurred, whichmay have at least partially cause the user to stop consuming the contentitem (for example, in order for the user to answer the door or phone).If there is no extrinsic environmental event detected, or if it isdetermined that a detected extrinsic environmental event was not a causeof the user's inattentiveness, the consumption monitoring system maydetermine that the user is not interested in the advertisement orcontent item. However, if an extrinsic environmental event is detected,the consumption monitoring system may determine that the user may stillbe interested in the content item and/or may determine another interestlevel of the user in the content item.

If the consumption monitoring system determines that the user's interestlevel in the content item is low (e.g., below a minimum interest levelthreshold), the consumption monitoring system may determine that theuser is likely not interested in the advertisement or content item andmay take some action to regain the user's attention, such as replacingthe advertisement or content item with another advertisement or contentitem for presentation to the user. In some instances, the consumptionmonitoring system may determine when the user is becoming lessinterested or engaged in the content and, in response, may take actionto regain the user's interest. The consumption monitoring system maytake some action such as selecting the replacement content item basedon, for example, a history of content item topics determined to be ofinterest to the user or the user's demographic group. The consumptionmonitoring system may verify whether the replaced content item hasobtained the user's attention and, if not, may repeatedly select adifferent replacement content item to present to the user and/or takeanother action until the consumption monitoring system presents acontent item that obtains the user's attention.

The consumption monitoring system may update a history of content itemtopics determined to be of interest to the user and/or a history ofcontent item topics determined to be of little or no interest to theuser. The user's history may be linked and/or otherwise associated withone or more demographic groups based on the user's demographiccharacteristics for use in determining which replacement content itemsare to be selected.

In some embodiments, a computing device may determine, duringpresentation of a first content item to a user, that the user is in anon-consumption state. In response to determining that the user is notconsuming content or is in the non-consumption state, the computingdevice may determine a user interest level in the first content itembased in at least in part on an event associated with thenon-consumption state. In response to a determination that the userinterest level meets a criterion, the computing device may cause asecond content item to be presented to the user.

In some embodiments, a computing device may monitor whether a user is ina non-consumption state in relation to a first content item beingprovided to a user. In response to determining that the user is in thenon-consumption state, the computing device may determine a timeframebased on a time at which the user entered the non-consumption state. Thecomputing device may capture activity data and environmental dataassociated with the user that occurred during the timeframe.

In some embodiments, a computing device may receive activity data andenvironmental data associated with a user. The computing device maydetermine whether the user is consuming a content item provided to theuser based on the activity data. In response to determining that theuser is not consuming the content item, the computing device maydetermine a user interest level in a topic of the content item based atleast in part on a response of the user to an environmental event.

These features in the summary above are merely illustrative of thefeatures described in greater detail below, and are not intended torecite the only novel features or critical features in the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

Some features herein are illustrated by way of example, and not by wayof limitation, in the accompanying drawings. In the drawings, likenumerals reference similar elements between the drawings.

FIG. 1 illustrates an example information access and distributionnetwork in accordance with one or more aspects as described herein.

FIG. 2 illustrates an example computing device that may be used toimplement any of the features and devices described herein.

FIG. 3 illustrates an example flow chart for performing a method inaccordance with one or more aspects as described herein.

FIG. 4 illustrates an example timeline of events and information inaccordance with one or more aspects as described herein.

FIGS. 5-8 illustrate example flow charts for performing methods inaccordance with one or more aspects as described herein.

DETAILED DESCRIPTION

In the following description of various illustrative embodiments,reference is made to the accompanying drawings, which form a parthereof, and in which is shown, by way of illustration, variousembodiments in which aspects of the disclosure may be practiced. It isto be understood that other embodiments may be utilized, and structuraland functional modifications may be made, without departing from thescope of the present disclosure.

FIG. 1 illustrates an example information distribution network 100 onwhich many of the various features described herein may be implemented.The information distribution network 100 may be any type of informationdistribution network, such as satellite, telephone, cellular, wireless,etc. One example may be a wireless network, an optical fiber network, acoaxial cable network, or a hybrid fiber/coax (HFC) distributionnetwork. Such networks use a series of interconnected communicationlinks 101 (e.g., coaxial cables, optical fibers, wireless, etc.) toconnect multiple premises 102 (e.g., businesses, homes, consumerdwellings, etc.) to a local office 103 (e.g., a headend, a processingfacility, etc.). The local office 103 may transmit downstreaminformation signals onto the links 101, and each premises 102 may have areceiver used to receive and process those signals.

There may be one link 101 originating from the local office 103, and itmay be split a number of times to distribute the signal to variouspremises 102 in the vicinity (which may be many miles) of the localoffice 103. The links 101 may include components not illustrated, suchas splitters, filters, amplifiers, etc. to help convey the signalclearly. Portions of the links 101 may also be implemented withfiber-optic cable, while other portions may be implemented with coaxialcable, other lines, or wireless communication paths.

The local office 103 may include a termination system (TS) 104, such asa cable modem termination system (CMTS) in an example of an HFC-typenetwork, which may be a computing device configured to managecommunications between devices on the network of links 101 and backenddevices such as servers 105-107 (to be discussed further below). In theexample of an HFC-type network, the TS may be as specified in astandard, such as the Data Over Cable Service Interface Specification(DOCSIS) standard, published by Cable Television Laboratories, Inc.(a.k.a. CableLabs), or it may be a similar or modified device instead.The TS may be configured to place data on one or more downstreamfrequencies to be received by modems at the various premises 102, and toreceive upstream communications from those modems on one or moreupstream frequencies. The local office 103 may also include one or morenetwork interfaces 108, which can permit the local office 103 tocommunicate with various other external networks 109. These networks 109may include, for example, Internet Protocol (IP) networks Internetdevices, telephone networks, cellular telephone networks, fiber opticnetworks, local wireless networks (e.g., WiMAX), satellite networks, andany other desired network, and the interface 108 may include thecorresponding circuitry needed to communicate on the network 109, and toother devices on the network such as a cellular telephone network, itscorresponding cell phones, web server 119, content source 126, etc.

As noted above, the local office 103 may include a variety of servers105-107 that may be configured to perform various functions. Forexample, the local office 103 may include a push notification server105. The push notification server 105 may generate push notifications todeliver data and/or commands to the various premises 102 in the network(or more specifically, to the devices in the premises 102 that areconfigured to detect such notifications). The local office 103 may alsoinclude a content server 106. The content server 106 may be one or morecomputing devices that are configured to provide content to users in thehomes. This content may be, for example, video on demand movies,television programs, songs, services, information, text listings, etc.The content server 106 may include software to validate (or initiate thevalidation of) user identities and entitlements, locate and retrieve (orinitiate the locating and retrieval of) requested content, encrypt thecontent, and initiate delivery (e.g., streaming, transmitting via aseries of content fragments) of the content to the requesting userand/or device.

The local office 103 may also include one or more application servers107. An application server 107 may be a computing device configured tooffer any desired service, and may run various languages and operatingsystems (e.g., servlets and JSP pages running on Tomcat/MySQL, OSX, BSD,Ubuntu, Redhat, HTML5, JavaScript, AJAX and COMET). For example, anapplication server may be responsible for collecting television programlistings information and generating a data download for electronicprogram guide listings. Another application server may be responsiblefor monitoring user viewing habits and collecting that information foruse in selecting advertisements. Another application server may beresponsible for formatting and inserting advertisements in a videostream and/or content item being transmitted to the premises 102.

An example premises 102 a may include an interface 110 (such as a modem,or another receiver and/or transmitter device suitable for a particularnetwork), which may include transmitters and receivers used tocommunicate on the links 101 and with the local office 103. Theinterface 110 may be, for example, a coaxial cable modem (for coaxialcable lines 101), a fiber interface node (for fiber optic lines 101), orany other desired modem device. The interface 110 may be connected to,or be a part of, a gateway interface device 111. The gateway interfacedevice 111 may be a computing device that communicates with theinterface 110 to allow one or more other devices in the home tocommunicate with the local office 103 and other devices beyond the localoffice. The gateway 111 may be a set-top box (STB), digital videorecorder (DVR), computer server, or any other desired computing device.The gateway 111 may also include (not shown) local network interfaces toprovide communication signals to other devices in the home (e.g., userdevices), such as televisions 112, additional STBs 113, personalcomputers 114, laptop computers 115, wireless devices 116 (wirelesslaptops, tablets and netbooks, mobile phones, mobile televisions,personal digital assistants (PDA), etc.), telephones 117, windowsecurity sensors 118, door home security sensors 119, tablet computers120, personal activity sensors 121, video cameras 122, motion detectors123, microphones 124, and/or any other desired computers, sensors,and/or other devices. Examples of the local network interfaces mayinclude Multimedia Over Coax Alliance (MoCA) interfaces, Ethernetinterfaces, universal serial bus (USB) interfaces, wireless interfaces(e.g., IEEE 802.11), Bluetooth interfaces, and others.

The local office 103 may include a content controller 125. Althoughshown separately, the content controller 125 may be physically and/orlogically combined with one or more of the push server 105, the contentserver 106 and the application server 107. The content controller 125may be or otherwise include a computing device responsible forinstructing a consumption monitoring system at a user premises 102 tomonitor a user's consumption state of a content item provided to theuser by a content presentation device (e.g., televisions 112, personalcomputer 114, laptop computer 115, tablet computer 120, audio musicdevice, or the like). The consumption monitoring system may be orotherwise include any device, or combination of devices, located atpremises 102 discussed herein. Alternatively, the consumption monitoringsystem may be located outside the premises 102, or be located inside andoutside the premises, in combination. For example, the consumptionmonitoring system may be or otherwise include an STB, DVR, tablet,personal computer, phone, tablet, gateway, modem, server, home alarmsystem, home control system, home monitoring system, sensors, such asmotion, sound, eye, or temperature sensors, and the like. A consumptionstate may correlate to the user's attentiveness in a content item asdetected by the consumption monitoring system. For example, adetermination by the consumption monitoring system (or by another devicein communication with the consumption monitoring system) that a user isnot attentively focusing on the content item (e.g., not paying attentionto the content item, not interacting with the content item, etc.) beingpresented may be referred to as a non-consumption state, whereas adetermination that the user is attentively focusing on the content itemmay be referred to as a consumption state. The content controller 125may also be responsible for determining a user's interest level in thecontent item in order to determine whether to instruct the contentpresentation device or another device controlling the contentpresentation device to present a different replacement content item inplace of the content item.

The consumption monitoring system may be used to collect informationthat may be informative to make a determination as to whether a user ispresently consuming (e.g., watching, listening to, and/or otherwiseconsuming) a content item, such as a movie, a television series, anadvertisement, a commercial, a video clip, an audio presentation, and/orthe like. The consumption monitoring system may be implemented using anydesired computing device, or combination of computing devices, such ascomputing device 200 discussed below, and may be (or be a part of), forexample, the gateway 111, the STB 113, the content presentation deviceitself such as the television 112, the personal computer 114, the laptopcomputer 115, the tablet computer 120, the smartphone 116, or the like.The consumption monitoring system may be communicatively connected to avariety of devices at the premises, such as any of elements 110-123 ofFIG. 1, and/or obtain information related to one or more servicesprovided to the user (e.g., a telephony service and/or an Internetservice).

The consumption monitoring system may collect information fordetermining the state of the user in any of a number of ways, and mayinclude or be communicatively connected to one or more devices (e.g.,sensors) in order to collect such information. For example, the videocamera 122 may provide a video feed and/or video-related information,and/or the microphone 124 may provide an audio feed and/or audio-relatedinformation, to the consumption monitoring system. Another sensor thatmay provide such information to the consumption monitoring system mayinclude, for example, the personal activity monitor 121. Many personalactivity monitors are commercially available and can provide informationabout the user's physical motion. Still other sensors may include, forexample, eye-tracking sensor devices, home security sensors such as thewindow security sensor 118, the door sensor 119, the motion detector123, and/or a home security system in general. The consumptionmonitoring system may further receive information from and/or concerningother devices such as the mobile device 116, the tablet 120, the phone117, the STB 113, the television 112, the personal computer 114, thelaptop computer 115, and/or the like. For example, the information mayinclude information about incoming and/or outgoing telephone calls viathe mobile device 116 and/or the telephone 117, information about webbrowsing activities via the personal computer 114, the laptop computer115, the mobile device 116, and/or the tablet 120, channel selectionand/or volume selection information via the STB 113 and/or thetelevision 112, etc. Where these devices include sensors such as motionsensors (e.g., many smart phones and tablets have miniature gyroscopes),location sensors (e.g., GPS and/or wi-fi triangulation), microphones,and/or video cameras, then information relating to these sensors mayalso be provided to the consumption monitoring system. One or more ofthe sensors may be part of the consumption monitoring system itself.

The consumption monitoring system may collect the information from theone or more devices and may also analyze at least some of theinformation to determine a state of the user, which may include anyinformation that may be relevant to whether the user is consuming thecontent, such as information indicating whether the user is detected asbeing present or not present, whether the user is facing the presentedcontent or facing away from the presented content (and/or facing anotherperson), whether the user's eyes are open or closed, the user'smovement, whether the user is talking and/or gesturing, the user'sposition (e.g., sitting, lying down, standing up, slouching), the user'slocation in a house (e.g., the room where the content is beingpresented, another identified room, etc.), whether the user is movingand/or the nature and/or timing of any movement, whether the user ismaking loud abrupt noises (such as yelling, cheering, or booing) orsofter noises, particular speech by one or more persons (e.g., viaspeech recognition technology), how many people are present in an areawhere the content is being presented, whether and/or in what manner theuser is using another device such as a phone (e.g., talking on thephone), a computer (e.g., browsing a particular website and/or enteringa particular search query), a state of another device (e.g., an incomingor outgoing call on the phone 117, a home security alarm state).

As an example, the user may be watching a movie being presented on thetelevision 112. At one or more points during the movie, information fromthe video camera 122, the microphone 124, and/or the personal activitymonitor 121 may be used to determine the state of the user. More thanone of each type of monitoring device can be used to increase accuracy,multiple views, etc. At one point, the consumption monitoring system maydetermine that the user is in a non-consumption state, perhaps becauseit is determined, based on the information, that the user is no longerconsuming the content and/or has fallen asleep. Or, while the user iswatching the movie, an incoming call is received by the phone 117, andinformation about the incoming call may be provided to the consumptionmonitoring system. The information about the incoming call may include,for example, the time of the call, the caller ID information of thecaller, which device receives the call, whether the call has beenanswered, how many rings occurred before the call was answered, and/orthe duration of the call. Shortly after the incoming call begins toring, the video camera 122 may provide information to the consumptionmonitoring system that may indicate that the user has gotten up, leftthe room, and/or picked up the phone 117. In the meantime, the personalactivity monitor 123 may also indicate movement consistent with such anactivity. Based on some or all of this information, the determined stateof the user may be that the user is not presently consuming contentand/or is no longer present. As will be described below, the consumptionmonitoring system or another device may also determine whether thenon-consumption state is due to a lack of interest in the content beingpresented or for another reason unrelated to the content beingpresented.

The consumption monitoring system may locally perform one or moredeterminations of whether the user is consuming a content item based ondata collected related to the user's attentiveness to the providedcontent item. For example, in response to determining that the user isin a consumption state, the consumption monitoring system may determineto continue monitoring the user's consumption state. Additionally, inresponse to determining that the user is in a non-consumption state(e.g., the user is not paying attention to content provided by thecontent presentation device), the consumption monitoring system maycollect information, generate a non-consumption state report. Thenon-consumption state report, or data associated with the report, may betransmitted to the content controller 125 so that the content controller125 may identify the content item, determine the user's interest levelin the content item, and/or determine whether to replace the contentitem, as discussed in detail in FIG. 3 below, for example.

While in the above examples, the consumption monitoring systemdetermined the consumption state, in some examples, the contentcontroller 125 at the service provider side may determine theconsumption state. As an example, the content controller 125 mayinstruct the consumption monitoring system to continuously transmitmonitoring information relating to the user, information associated withthe provided content item, and the user's environmental information,upstream to the content controller 125. In response to determining thatthe user is in a consumption state, the content controller 125 maycontinue to monitor the user's consumption state. In response todetermining that the user is in a non-consumption state, the contentcontroller 125 may generate a timestamp associated with thenon-consumption state. As discussed in further detail below, the contentcontroller 125 may identify the content item, determine the user'sinterest level in the content item, and determine whether to replace thecontent item. Replacing the content may comprise adjusting the contentitem to another version of the content item (e.g., changing a scene,changing the language or rating version, etc.).

As used herein, when one computing device instructs a differentcomputing device to perform one or more operations, the computing devicemay send one or more instructions with one or more parameters for theoperations to the different computing device, which may execute the oneor more instructions with the one or more parameters to perform theinstructed operations.

FIG. 2 illustrates general hardware elements of an example computingdevice 200 that can be used to implement any of the elements discussedherein and/or illustrated in the figures. The computing device 200 mayinclude one or more processors 201, which may execute instructions of acomputer program to perform any of the features described herein. Theinstructions may be stored in any type of computer-readable medium ormemory, to configure the operation of the processor 201. For example,instructions may be stored in a read-only memory (ROM) 202, a randomaccess memory (RAM) 203, a removable media 204, such as a UniversalSerial Bus (USB) drive, compact disk (CD) or digital versatile disk(DVD), floppy disk drive, or any other desired electronic storagemedium. Instructions may also be stored in an attached (or internal)storage 205 (e.g., hard drive, flash, etc.). The computing device 200may include one or more output devices, such as a display 206 (or anexternal television), and may include one or more output devicecontrollers 207, such as a video processor. There may also be one ormore user input devices 208, such as a remote control, keyboard, mouse,touch screen, microphone, camera, etc. The computing device 200 may alsoinclude one or more network interfaces, such as input/output circuits209 (such as a network card) to communicate with an external network210. The network interface may be a wired interface, wireless interface,or a combination of the two. The interface 209 may include a modem(e.g., a cable modem), and network 210 may include the communicationlinks 101 discussed above, the external network 109, an in-home network,a provider's wireless, coaxial, fiber, or hybrid fiber/coaxialdistribution system (e.g., a DOCSIS network), or any other desirednetwork.

The FIG. 2 example is an example hardware configuration. Modificationsmay be made to add, remove, combine, divide, etc. components as desired.Additionally, the components illustrated may be implemented using basiccomputing devices and components, and the same components (e.g.,processor 201, ROM 202, etc.) may be used to implement any of the othercomputing devices and components described herein. For example, thevarious components herein may be implemented using computing deviceshaving components such as a processor executing computer-executableinstructions stored on a computer-readable medium, as illustrated inFIG. 2.

One or more aspects of the disclosure may be embodied in computer-usabledata and/or computer-executable instructions, such as in one or moreprogram modules, executed by one or more computers (such as computingdevice 200) or other devices to perform any of the functions describedherein. Generally, program modules include routines, programs, objects,components, data structures, etc. that perform particular tasks orimplement particular abstract data types when executed by a processor ina computer or other data processing device. The computer executableinstructions may be stored on one or more computer readable media suchas a hard disk, optical disk, removable storage media, solid statememory, RAM, etc. The functionality of the program modules may becombined or distributed as desired in various embodiments. In addition,the functionality may be embodied in whole or in part in firmware orhardware equivalents such as integrated circuits, field programmablegate arrays (FPGA), and the like. Example data structures may be used toillustrate one or more aspects described herein, but these are merelyillustrative examples.

FIG. 3 illustrates an example flow chart for performing a method inaccordance with one or more aspects as described herein. The method ofFIG. 3 and/or one or more steps thereof may be performed by a computingdevice (e.g., an STB, a gateway, a headend device, etc.). The methodillustrated in FIG. 3 and/or one or more steps thereof may be partiallyor fully embodied, for example, in computer-executable instructions thatare stored in a computer-readable medium, such as a non-transitorycomputer-readable memory. In some instances, one or more of the steps ofFIG. 3 may be performed in a different order and/or combined. In someinstances, one or more of the steps of FIG. 3 may be omitted and/orotherwise not performed.

As seen in FIG. 3, the method may begin at step 301 in which a computingdevice (e.g., the consumption monitoring system) may collect a user'sactivity and environmental data from a variety of computing devices,monitors, sensors, etc. The user's activity and environmental data mayinclude, for example, the location of the user (e.g., location within aroom, identification of which room the user is in, location relative tothe content presentation device, etc.), orientation of the user (e.g.,facing or away from the content presentation device), the posture of theuser, body movement of the user, heart rate and/or other biologicalsigns of the user, eye movement of the user, speech, and/or otheractivities of the user, which may be informative as to whether the useris present and attentive to the content item (e.g., watching the contentitem, talking about the content item, or the like). The user activityand environmental data may also include any information about theenvironment surrounding the user (e.g., that the doorbell rang, that thephone rang, whether a tweet or text message was received or sent by theuser, what web page is being browsed by the user, whether keyboardtyping sounds are detected via a microphone, whether a home alarm wastriggered, the temperature in the room, sounds emitted from sourcesother than the user, appliance use data (e.g., information about theusage of devices at the user premises), the presence or movement of oneor more other persons or animals in the room or in another room, theuser's calendar appointments, etc.).

In one example of implementing step 301, the consumption monitoringsystem may begin collecting the user's activity and environmental datain response to one or more triggering events defined in the instructionssent by the content controller 125. One triggering event may correspondto, for instance, when one or more content items are provided by acontent presentation device to the user. As an example, the consumptionmonitoring system may begin collecting the data when an image or videostarts being displayed by the content presentation device and/or whenaudio is output by a speaker. In one or more arrangements, the contentcontroller 125 may transmit instructions to the consumption monitoringsystem to monitor a user's consumption of the one or more content itemsprovided by the content presentation device. As discussed herein, acontent item may include, for example, an image, video (e.g., a movie,television program, etc.), audio feed (e.g., music) and/or the like. Thecontent presentation device may be or otherwise include, for example, atelevision, personal computer, laptop computer, tablet computer, stereosystem, gaming console, smartphone, mobile phone, or the like.

The consumption monitoring system may collect the user's activity dataand environmental data from a variety of sensors of one or morecomputing devices at the user's location. One or more sensors may bepart of the consumption monitoring system or may be part of one or moreremote devices (also referred to herein as remote sensor devices)communicatively connected with the consumption monitoring system. Asensor may be part of, for example, the display device (e.g.,television), entertainment/gaming console, gaming console, home securitysystem, and devices typically carried on the person such as smartphones,smartwatches, fitness trackers, health trackers, etc. If a sensor ispart of the consumption monitoring system, the consumption monitoringsystem may directly obtain activity and/or environmental data from thesensor. If a sensor is remote from the consumption monitoring system,the consumption monitoring system may discover the computing devices byscanning for devices connected to a particular network and/or by manualinput from the user or operator of the headend.

Further, the consumption monitoring system may transmit instructions tothe remote sensor devices to instruct the sensor devices to begin toperiodically and/or continuously obtain data relating to the user and/orhis or her environment via its one or more sensors and record such datain a recordation log for periodic transmission to the consumptionmonitoring system (e.g., a push relationship). In addition oralternatively to the push relationship, a pull relationship may be used.Namely, the consumption monitoring system may query the remote sensordevice for its data and, in response, may receive the sensor's data fromthe remote sensing device. Accordingly, a sensor may transmit itssensing data to the consumption monitoring system either periodically onits own or in response to a query from the consumption monitoring systemor in response to an action it detected. The transmission may be in theform of a message comprising the data from the sensor and an identifierof the transmitting remote sensor device. The identifier may be a mediaaccess control (MAC) address or other identifier of the remote sensordevice.

Additionally, the message may also include one or more timestamps ofwhen the sensing data was generated and/or otherwise obtained by thesensor device. The message may include, for example, the time periodsince the computing device has last transmitted its sensing data to theconsumption monitoring system. A history of sensing data in therecordation log may also be transmitted to the consumption monitoringsystem. The history of sensing data may include sensing data that wasgenerated prior to the current/recent sensing data (e.g., all senseddata within the last two hours, or since yesterday, etc.). Each type ofsensor will be discussed in detail below in conjunction with step 302.

In step 302, a computing device (e.g., the consumption monitoringsystem, the content controller 125, etc.) may, for example in real-time,determine whether the user is presently consuming a content itemprovided by the content presentation device based on the user's activityand environmental data. As discussed herein, when the user is presentlyconsuming a content item, the user may be said to be in a consumptionstate. Similarly, when the user is not presently consuming a contentitem, the user may be said to be in a non-consumption state.

As noted above, the consumption monitoring system may collectinformation from a variety of sensors in step 301 and determine whetherthe user is in a consumption or non-consumption state in step 302. Thesensors may include a variety of different types of sensing devices toaid in determining whether the user is currently consuming a providedcontent item. One or more of the sensors may determine location data ofa computing device that may be physically on the user's person, such aspersonal activity monitors (PAMs), smartwatches, mobile phones, etc.,and which may be used to approximate the user's location. For example, acomputing device (e.g., a smartphone) may include a global positioningsystem (GPS) sensor that may either locally compute and/or otherwiseobtain from a remote device the computing device's GPS coordinates(e.g., a set of latitude and longitude coordinates) or other locationidentifying information. The computing device may determines itslocation identifying information using triangulation with other devices(e.g., cell phone triangulation, Wi-Fi triangulation), detection of abeacon from a particular device, etc. The computing device may send theGPS coordinates or other location identifying information to theconsumption monitoring system along with a timestamp of when each set ofGPS coordinates was computed and/or otherwise determined by thecomputing device. Using this information, the consumption monitoringsystem may determine the location of the computing device, which is thelikely location of the user. The consumption monitoring system maycompare the computing device's location with the known location of thecontent presentation device to determine whether the user is closeenough to the content presentation device to actually be able to consumea content item by determining a distance between the computing deviceand the content presentation device. In some instances, the distancebetween the computing device and the content presentation device may bedetermined using audible sounds. For example, the computing device mayinclude a microphone that may receive audible sounds from the contentpresentation device and may estimate the distance based on the volume ofthe audible sounds. If the distance between the computing device and thecontent presentation device is less than or equal to (or alternatively,is simply less than) a maximum threshold distance, the consumptionmonitoring system may determine that the user is close enough to be ableto consume the content items provided by the content presentationdevice. Otherwise, if the distance between the computing device and thecontent presentation device is greater than (or alternatively, isgreater than or equal to) the maximum threshold distance, theconsumption monitoring system may determine that the user is not closeenough to be able to consume the content items provided by the contentpresentation device. The maximum threshold distance may be predeterminedor it may be determined dynamically based on one or more factors. Thisis only one example of how the consumption monitoring system maydetermine whether a user is close enough to consume a presented contentitem based at least in part on location identifying information, e.g.,the location of the computing device, the location of the contentpresentation device, and/or a distance between the two devices.

Additionally or alternatively, the location identifying information mayinclude one or more received signal strength indications (RSSIs). Theconsumption monitoring system may continuously or intermittently (e.g.,periodically) query a computing device typically on a user's person(e.g., smartwatch, smartphone, etc.) and, in response, may receive aresponse from the computing device via a network (e.g., a Wi-Finetwork). The consumption monitoring system may determine an RSSI valuebased on the received response (for instance, either the signal strengthof the response or an RSSI value in the response itself) and use theRSSI value to approximate the distance between the consumptionmonitoring system and the computing device. For example, a higher RSSIvalue may be associated with stronger signal strength of the receivedresponse, which may indicate that the computing device is located inclose proximity to the consumption monitoring system. Conversely, alower RSSI value may be associated with lower signal strength of thereceived response, which may indicate that the computing device islocated farther away from the consumption monitoring system. If the RSSIvalue is below a minimum threshold RSSI value (or alternatively, belowor equal to the minimum threshold RSSI value), the consumptionmonitoring system may, for example, determine that the user is notpresently consuming a content item since the user is likely not in thesame room as the content presentation device. The response may include,for example, a network identifier of the network used to transmit theresponse and/or a timestamp of when the response was sent to theconsumption monitoring system.

One or more of the sensors that provide input to the consumptionmonitoring system may be accelerometers, gyroscopes and/or other motiondetectors, which may be included within one or more computing devices(e.g., PAMs, smartphones, smartwatches, etc.). These computing devicesmay send motion data (e.g., acceleration information, velocityinformation, direction information) of the computing device to theconsumption monitoring system along with one or more timestamps of whenthe motion occurred. Because the user is moving the computing device,such motion data may be used to determine the user's excitement orenergy level which may be useful in determining a user's interest levelin a content item. Thus, a user's consumption or non-consumption statemay be determined at least in part on the motion data. For instance, ifthe user is moving the computing device faster than a preset thresholdspeed/velocity, the consumption monitoring system may determine that theuser is excited. If the user is moving the computing device slower thanthe preset threshold speed/velocity, the consumption monitoring systemmay determine that the user is awake but not excited. If the user hasnot moved the computing device for at least a preset time period, theconsumption monitoring system may determine that the user is asleep and,thus, not consuming the presented content items.

One or more of the sensors may be heart rate monitors and/or othersensors that measure a bodily function of the user, which may beincluded within a variety of computing devices such as smartphones,personal activity monitors (PAM), health monitors, etc. The computingdevices may send to the consumption monitoring system a measurement of abodily function of the user (e.g., the user's heartrate) and a timestampof when the measurement was taken. The consumption monitoring system mayuse the measurements to determine an energy or excitement level of theuser. Thus, a user's consumption or non-consumption state may bedetermined based at least in part on bodily function measurement data.For example, if the user's heartrate is greater than a first presetthreshold, the consumption monitoring system may determine that the useris excited. If the user's heartrate is less than the first presetthreshold but greater than a second preset threshold, the consumptionmonitoring system may determine that the user is awake but not excited.If the user's heartrate is less than the second preset threshold, theconsumption monitoring system may determine that the user is asleep andthus not consuming the presented content item.

One or more sensors may be optical sensors that capture the user'sphysical position in an image or video. The consumption monitoringsystem may receive imaging data of the user's position from opticalsensors (e.g., video recorders, cameras, or the like) and a timestamp ofwhen the image or video was taken. The imaging data (e.g., images and/orvideo) may be of an area near the content presentation device (e.g., anarea immediately in front of a television, computer, tablet, etc.). Theconsumption monitoring system may determine whether the user is in aconsumption state based at least in part on the imaging data. Theconsumption monitoring system may, using the received imaging data,determine how many people are present in the room where the contentitems are being presented and whether each of them is physically in aposition such that they can view the presented content item. Thus, auser's consumption or non-consumption state may be determined based atleast in part on the imaging data. For instance, the consumptionmonitoring system may determine the direction each person is facing(e.g., whether a person is facing away from or towards the television)and whether each person's eyes are open or closed and for what duration(e.g., a brief wink, a nap, etc.). The consumption monitoring systemmay, for example, use eye-tracking algorithms to determine the objectseach person is viewing. For example, the consumption monitoring systemmay determine if a person is looking at the television, smartphone,tablet, computer, or any other object. In instances where the person islooking at a screen of the display device, the consumption monitoringsystem may determine what part of the screen the person is viewing orwhat content item on the screen the person is viewing. In someinstances, the consumption monitoring system may identify each person inthe room using person identification technologies such as facialrecognition. Thus, the consumption monitoring system may determinewhether each person present in the room is actually being attentive tothe presented content item.

One or more sensors may be a microphone or other audio input devices torecord audible sounds produced by the user (e.g., a conversation withanother user, a cheer, etc.). The consumption monitoring system mayinclude or be communicatively connected to a speech recognition module,which may identify words uttered by the user based on audio dataprovided from the audio input device(s). The consumption monitoringsystem may determine a topic of conversation using the uttered words andmay also determine a topic of the content item by either receiving anindication of content item's topic from the headend 103 or by locallydetermining content item's topic using e.g., closed captioninginformation. The consumption monitoring system may compare the topicuttered by the user with the content item's topic to determine whetherthe user is speaking about the content item. If the topics match, theconsumption monitoring system may determine that the user is consumingthe content item. Otherwise, if the topics do not match, the consumptionmonitoring system may determine that the user is not consuming thecontent item. The consumption monitoring system may, for example,determine that the user is not consuming the content item if the volumelevel of the content item is below a minimum volume level threshold(e.g., the user has muted the television program). Specific words, avolume level, and/or other characteristics of detected sound (e.g.,emitted from the content presentation device, from the user, and/or fromthe user's environment) may be associated with different level ofexcitement of the user and/or with other user status. For example, aloud cheer may be associated with a high excitement level of the userfor the content item. Thus, the user's consumption or non-consumptionstate may be determined at least in part on the audio data.

In addition to the user's physical position and posture, user activityand environmental data may also include any other data related to theuser's actions. As an example, user activity and environmental data maybe an indication of a webpage (e.g., a URL) the user is visiting on acomputing device such as a tablet. As another example, user activity andenvironment data may include telephonic information. For instance, thegateway 111 may intercept incoming (or outgoing) telephone calls anddetermine who is calling, whether the phone was answered, and, if so,the duration of the conversation. In some instances, the local office orgateway 111 may determine what content each party to the telephone callis viewing and whether they are viewing the same content. As yet anotherexample, user activity and environmental data may include whether a dooror window was opened, which may be received from the home securitysystem. As still yet another example, user activity and environmentaldata may include appliance use data, which may be received from a smartappliance. For instance, a smart refrigerator may send a timestamp ofwhen one of its doors has been opened. For instance, a microwave maysend a timestamp of when its door has been opened.

The user activity and environmental data may include information from ahome security system. For example, the home security system maytransmit, to the consumption monitoring system, information relating towhether a security alarm has been activated, whether a window or doorhas been opened, whether the doorbell was rang, whether the garage doorhas been opened, and/or the like. The information may also include oneor more timestamps of when the alarm was triggered, door was opened,window was opened, etc.

The user activity and environmental information may include, forexample, communication information from a variety of computing devices.The gateway 111 may intercept and analyze various communications toand/or from the computing devices. The communication information mayinclude, for example, requests for a webpage or query for a web-basedsearch engine. The communication information may include telephoniccommunication information received from a phone or the gateway 111. Thetelephonic communication information may include the time at which aphone (e.g., home phone, smartphone, or other phone) rang and anidentifier of the person attempting to contact the user. As an example,the telephonic information may include an indication that the user'shome phone rang at 2:00 pm and the call was from the user'smother-in-law. Other communication data may include email notifications,instant messages, requests for IP-enabled voice chats, text messages,mobile app specific communications, etc. and may include similarinformation to that discussed above for a telephonic call. The gateway111 may transmit the intercepted information to the consumptionmonitoring system. Alternatively, the gateway 111 may itself be orotherwise comprise the consumption monitoring system.

As mentioned above, the user activity and environmental data may includeinformation received from smart appliances. In one example, a microwavemay send a timestamp of when the microwave outputted an audible alarm(e.g., beeped) to signal that the food is done cooking.

The consumption monitoring system may determine whether the user is in aconsumption state or a non-consumption state based on various useractivity and/or environmental data locally or remotely obtained from oneor more of the above-described sensor devices. The consumptionmonitoring system may have received preset rules of combinations ofconditions that lead to a determination of consumption state or anon-consumption state. As will be explained below, the rules may alsodynamically change over time based on learned user habits (which may bebased on the user activity and/or environmental data), such as via acomputer learning algorithm.

If the consumption monitoring system determines that the user is in aconsumption state (e.g., is presently watching or listening to a videoprogram on the television), the consumption monitoring system maycontinue to repeat the steps of 301 and 302 to repeatedly collect moredata relating the user and his or her environment as well as repeatedlydetermine whether the user is still in a consumption state.

In response to determining that the user is in a non-consumption state(e.g., that the user is not watching and/or listening to a displayedcontent item), the consumption monitoring system may, in step 303,determine a timestamp of the user's non-consumption state and correlaterelevant user activity and/or environmental data with the user'snon-consumption state. The consumption monitoring system may generate atimestamp of when the consumption monitoring system determined that theuser entered into a non-consumption state. A timestamp, as discussedherein, may be in any format. For example, a timestamp may includemultiple fields to account for year, month, day, hour, minute, seconds,tenths of seconds, am or pm, day of the week, etc. (e.g., Jun. 16, 2014at 1:12 pm). The consumption monitoring system may, for instance, use aninternal clock to determine the timestamp for the instance of the user'snon-consumption state. The internal clock may be synchronized with aclock of the headend used in transmitting the data stream (e.g., an edgeserver) to ensure that the timestamp is accurate as to both theconsumption monitoring system and the headend. As a result, thetimestamp may be used either by the headend without needing to adjustfor clock drift. The consumption monitoring system may also identify thecontent item. In some instances, the consumption monitoring system maydetermine an identifier of the content item itself. In other instances,the consumption monitoring system may determine an identifier of thestream that transported the content item (e.g., channel 5) so that theheadend may use the timestamp of the user's non-consumption state andthe stream identifier to determine the content item that was beingprovided to the user at that time.

The instructions received from the headend may include a dynamictimeframe for use in determining which data from the remote sensorsshould be correlated with and/or is otherwise relevant in determiningthe reasons for the user's non-consumption state. The dynamic timeframemay have an end time equal to (or otherwise based on) when the userentered a non-consumption state. Using the dynamic timeframe and thetimestamp of when the user entered a non-consumption state, theconsumption monitoring system may determine the start time of the presetdynamic timeframe. As an example, the dynamic timeframe may be alook-back timeframe of a preset amount of time such as 10 seconds beforethe present time. If the user entered a non-consumption state at 10:30am and 30 seconds, the consumption monitoring system may identify thistime as the end time of the dynamic timeframe and determine that thestart time of the dynamic time period is 10:30 am and 20 seconds.Accordingly, the relevant time period may be from 10:30 am and 20seconds to 10:30 am and 30 seconds. The consumption monitoring systemmay correlate and/or otherwise link user activity and/or environmentaldata having a timestamp within this time period to the user'snon-consumption state to effectively take a “snapshot” of the userand/or his or her environment just before the user stopped consuming thecontent item so that either consumption monitoring system or a headenddevice may determine a possible reason for the user's non-consumptionstate. As an example, if the doorbell rang within this time period, suchinformation would be correlated with the user's non-consumption state.The duration of the dynamic timeframe may be any amount of time, forexample, 1 second, 5 seconds, 10 seconds, 30 seconds, 1 minute, or more,or any amount of time in between these values.

The end time of the dynamic timeframe may be the time when the userentered a non-consumption state. Or, the end time may be a time otherthan, yet based on or otherwise associated with, the time when the userentered the non-consumption state, such as a preset duration (e.g., 5seconds, 15 seconds, etc.) after the user entered the non-consumptionstate. As an example, if it is determined that the user entered anon-consumption state at 10:30 and 30 seconds, the start time may be10:30 and 20 seconds and the end time may be 10:30 and 40 seconds. Thepreset duration after the user enters the non-consumption may enable theconsumption monitoring system to continue to collect user activityand/or environmental data and correlate that data with the user'snon-consumption state. While the duration in the above examples ispreset, the duration may alternatively be dynamically chosen and, forexample, be a different duration that depends upon the user activitydata and/or the environmental data received by the content consumptiondevice. By continuing to collect the data after the user enters anon-consumption state, the consumption monitoring system may be able toobtain data that may reflect why the user entered a non-consumptionstate. As an example, if the user leaves his seat (entering anon-consumption state) immediately after the doorbell rang, theconsumption monitoring system may be able to detect whether the useranswered the door a few seconds later.

The consumption monitoring system may generate a non-consumption statereport based on the data collected. The non-consumption state report mayinclude, for example, the timestamp of the non-consumption state, thecontent item identifier, the stream identifier, the data obtained fromthe sensor devices that have been correlated with the user'snon-consumption state (e.g., data that has a timestamp within the presetdynamic time period), and the data's timestamps. This data may includelocal uncontrollable events (e.g., phone calls, doorbells, etc.) thatoccurred just before the user entered into a non-consumption state. Thenon-consumption state report may include an indication of the start timeand the end time of the relevant timeframe.

FIG. 4 illustrates an example timeline of events and information for anon-consumption state report in accordance with one or more aspects asdescribed herein. As an example, the user may be watching a movie beingpresented on television 112. The consumption monitoring system maycollect user activity and/or environmental data from a variety of sensordevices including, for example, video camera 122, microphone 124, phone117, and/or personal activity monitor (PAM) 121 (step 301). Theconsumption monitoring system may determine that the user is consumingthe content item (e.g., the movie has the user's full attention) upuntil time T1 (steps 301 and 302). At time T1, the consumptionmonitoring system may determine that that the user has answered anincoming call (e.g., a local uncontrollable event) and is now in anon-consumption state since the user is both not physically present nearthe content presentation device (e.g., within a preset distance) and isparticipating in the telephone call (step 302). The consumptionmonitoring system may determine the timestamp of the user'snon-consumption state (e.g., T1) and correlate relevant user's activityand/or environmental data (step 303). In this example, the environmentaldata may include ringing of the phone 117 as a result of the incomingcall, and the user activity data may include an indication that the useranswered the call, each of which may be included in the non-consumptionstate report.

Returning to FIG. 3, in step 304, a computing device (e.g., theconsumption monitoring system, the content controller 125) may determineone or more reasons for the user's non-consumption state and the user'sinterest scores in a topic of the content item. This step may beperformed, for example, locally by the consumption monitoring systembased on instructions received from the content controller 125. Or, thisstep may be performed, for example, by the content controller 125 oranother device upon receiving the non-consumption report from theconsumption monitoring system.

In order for the computing device to obtain the user's interest scoresin a content item, the computing device may perform the illustrativemethod depicted in FIG. 5. The user's interest score may be based atleast in part on the user activity and environmental data includinglocal uncontrollable events that may have diverted the user's attention.The method of FIG. 5 and/or one or more steps thereof may be performedby a computing device (e.g., content controller 125). The methodillustrated in FIG. 5 and/or one or more steps thereof may be embodied,for example, in computer-executable instructions that are stored in acomputer-readable medium, such as a non-transitory computer-readablememory. In some instances, one or more of the steps of FIG. 5 may beperformed in a different order and/or combined. In some instances, oneor more of the steps of FIG. 5 may be omitted and/or otherwise notperformed. While the below steps with respect to FIG. 5 will bedescribed with respect to being performed by the content controller 125,the steps of FIG. 5 may be performed by the consumption monitoringsystem and/or one or more other devices.

As seen in FIG. 5, the method may begin at step 501 in which the contentcontroller 125 may obtain a non-consumption state report (discussedabove) from the consumption monitoring system. Further, the contentcontroller 125 may also determine a topic of the content item that wasbeing provided when the user entered a non-consumption state. In someinstances, the non-consumption state report itself may include anidentifier of the content item (e.g., a program identifier, title,etc.). In further instances, the non-consumption state report mayinclude an identifier of the stream so that the content controller 125may determine the content item's identifier by looking up the contentitem in a program schedule database using the stream identifier and thetimestamp of the user's non-consumption state. In either instance, thecontent controller 125 may use the content item's identifier todetermine the topic of the content item by querying a databasecorrelating topics to content items. Additionally, the contentcontroller 125 may determine a current interest score in the topic ofthe content item. In some instances, the content controller 125 mayassign a default interest score when, for example, the contentcontroller 125 does not find a user's past interest score for the topic.In further instances, the content controller 125 may query a database,using an identifier of the user and an identifier of the topic, todetermine a user's most-recent past interest score for the topic (e.g.,determined from a previous analysis using of a user's non-consumptionstate for a different content item having the same topic) and mayidentify that interest score as the user's current interest score in thetopic of the content item.

In step 502, the content controller 125 may, for the relevant timeframe(e.g., the determined dynamic timeframe), determine whether there areone or more corresponding local uncontrollable events using the datafrom the non-consumption state report. A local uncontrollable event maybe one or more events that occurred in the user's environment but mightnot have been initiated by the user. As an example, a ringing doorbellor phone may be considered a local uncontrollable event. If the contentcontroller 125 determines that there were one or more uncontrollableevents, then, in step 503, it may determine whether the localuncontrollable event(s) likely caused the user's non-consumption state.Following the doorbell example, if the non-consumption report alsoincludes an indication that the user answered the door in the relevanttimeframe, then the content controller 125 may determine that the user'snon-consumption state was caused by the uncontrollable event and, instep 504, may increase (or, alternatively, maintain) the user's currentinterest score in the topic of the content item since the uncontrollableevent caused the user's attention to be diverted from consuming thecontent item. As an example, if the user's current interest score in thetopic is 7 out of 10 (e.g., a maximum score). The content controller 125may increase the score to 8, 9, 10, etc. and may store the increasedscore in a topic database. However, there may be one or more exceptionswhere the user's interest score should be decreased rather thanincreased in response to determining that the uncontrollable eventcaused the user's non-consumption state if, for example, the contentcontroller 125 determines that the user typically ignores a particularuncontrollable event, which may be stored in a habits database in theheadend. As an example, if the user habitually does not answer the phonewhen his or her mother-in-law calls, but the user does in fact answerthe phone, this may indicate that the user is not interested in thetopic of the content item and, as a result, the content controller 125may reduce the user's interest score in the topic of the content item.

Otherwise, if either there were no local uncontrollable events or if theuncontrollable events did not likely cause the user's non-consumptionstate, the content controller 125 may, in step 505, decrease the user'scurrent interest score in the topic of the content item. For example,the content control 125 may decrease the user's current interest scorein the topic from 7 out of 11 to 5 or 6 out of 11 since nouncontrollable event caused the user's non-consumption state which mayindicate that the user is not interested in the topic.

Additionally, the user's interest score in the topic of the content itemmay be modified (increased or decreased) based on the user's actions. Asdiscussed above, the non-consumption state report might not only includeenvironmental data used in determining local uncontrollable events butmay also include user activity data. As an example, a user may haverequested, searched for, and/or otherwise is interacting with a webpageduring the relevant timeframe. If the topic of the webpage matches thetopic of the content item, the content controller 125 may increase theuser's interest score in the topic. If not, the content controller 125may decrease the user's interest score in the topic. As another example,if the user's heartbeat is above a preset threshold during a preset timeperiod (e.g., a few seconds) before the user entered a non-consumptionstate, the content controller 125 may determine that the user is excitedabout the topic and may increase the user's interest score in the topic.As yet another example, if the user is cheering about the topic, thecontent controller 125 may increase the user's interest score in thetopic.

Following the example depicted in FIG. 4, the content controller 125 maydetermine, for the relevant timeframe around time T1, that the userexperienced an uncontrollable event (e.g., the ringing of the phone 117for the incoming call) (steps 501 and 502). The content controller 125may also determine that the incoming call caused the user'snon-consumption state based on the indication that the user answered thecall (step 503). The content controller 125 may increase or maintain theuser's interest score in the topic of the content item provided to theuser at time T1 (unless the user habitually ignores calls from thisperson, in which case the user's interest score in the topic of thecontent item provided at time T1 may be reduced).

Returning to FIG. 3, if the content controller 125 determines that theuser's interest score is below a minimum interest score threshold, thecontent controller 125 may optionally perform step 305 by taking actionto regain the user's attention. For example, if the minimum interestsscore threshold is 7 and the user's interest score is 6, the contentcontroller 125 may determine that the user's interest score is below theminimum interest score threshold. In some instances, the minimuminterest score threshold may vary based on time of day, day of week,etc. For example, during the morning, the user may be getting ready forwork and have the television on in the background. As a result, duringthe morning, the minimum interest score threshold may be set to 3. Inthe evening (e.g., during primetime), the minimum interest threshold maybe set to 8. In some instances, the minimum interest score threshold mayvary based on content (e.g., a user's favorite show may have a higherminimum interest score threshold than shows which are being rerun).

The content controller 125 may then select another content item having adifferent topic to replace the content item currently being provided tothe user. The content controller 125 may select a topic for which theuser has an interest score that is above the minimum score thresholdand, optionally, the topic that has the user's highest interest score byquerying a database containing the user's topic scores and correspondingtopics using an identifier of the user and performing the appropriateselection. The content controller 125 may then select a content itemhaving the selected topic from a set of multiple content items toreplace or supplement the content item currently being provided to theuser and for which the user might not have an interest in so that theuser's attention may be regained. The content controller 125 may alsoselect the content item based on the user's commonality or demographicassociation with one or more groups discussed more in further detailbelow in step 306. In one example, an advertisement or commercial forfeminine skin products currently being displayed to the user may bereplaced with an advertisement or commercial for car products. Forinstance, the content controller 125 instruct another device (e.g.,content server) to provide the replacement content item (e.g., thereplacement advertisement or commercial) to the content presentationdevice, a controller (e.g., set-top box, etc.) of the contentpresentation device, and/or the consumption monitoring system to forwardto the content presentation device so that the content presentationdevice may provide and/or otherwise output (e.g., display) thereplacement content item to the user. In some cases, the contentcontroller 125 may retrieve the replacement content item and provide thecontent item to the appropriate device.

The process of steps 301-305 may be repeated until the user's attentionis regained or has an interest score that is above the minimum scorethreshold. If the replacement content item did regain the user'sattention, the consumption monitoring system may take some action, suchas informing the content controller 125 of such so that the contentcontroller 125 may, in response, increase the user's interest levelscore in the topic of the replacement content item. If the contentcontroller 125 determines that the user's interest score in topic of thecontent item currently being provided is greater than or equal to theminimum interest score threshold, the content controller 125 might nottake action to regain the user's attention thereby continuing to providethe currently-provided content item via the content presentation device.

Following the example of FIG. 4, the content presentation device maypresent a replacement content item that regains the user's attention(step 305). The consumption monitoring system may repeatedly collectuser activity data and/or environmental data from the sensor devices(step 301) and repeatedly determine whether the user is consuming thecontent item (step 302). The consumption monitoring system may presentor change the content if attention is lowered and may also monitor ifattention increases as a result of the changed content. As timeprogresses, different content items may be presented to the user. Inthis example, a different content item may be presented at time T3 atwhich point the consumption monitoring system determines that the useris present but reading e.g., via an optical sensor. The consumptionmonitoring system may generate a non-consumption state report and sendit to the content controller 125, which may then determine if there wasa local uncontrollable event. Since there was not a local uncontrollableevent, the content controller 125 may reduce the user's interest scorein the content item provided at time T3. Alternatively, in someinstances, the content controller 125 may reduce the user's interestscore in the topic because the user is reading a webpage having a topicthat does not match the topic of the content item. In other instances,the content control 125 may increase the user's interest score in thetopic because the user is reading a webpage having a topic that matchesthe topic of the content item.

In step 306, the content controller 125 may determine commonalityamongst users (e.g., in various user premises 102) and/or demographics.The content controller 125 may form many groups based on one or moredemographic characteristics of users (e.g., age, gender, etc.) and storethe groupings in a demographic database. The content controller 125 mayassign the user to one or more groups that have one or more matchingcharacteristics to that of the user. Within each grouping, one or moretopics may be associated with a corresponding interest score for thegroup that is updated based on each member's interest score in thattopic (e.g., an average of interest score in the topic) for use inselecting replacement content items.

In step 307, the content controller 125 may use the groups' interestscore in a topic when determining/modifying future content (e.g., whendetermining what replacement or supplemental content items (e.g.,advertisements) to display to the user in the future). The contentcontroller 125 may also learn the habits of the user based on the user'sactivity and/or environmental data, and store the user's habits in thehabits database discussed above in steps 304 and 504. For instance, ifthe user typically does not pick up calls from his or her mother-in-law,the content controller 125 may store the user's habit of ignoring thisparticular uncontrollable event in the habit database.

Additionally, the content controller 125 may use the group's interestscores in topics in negotiating advertisement slots. FIG. 6 depicts anillustrative method for using interests score in accordance with one ormore aspects discussed herein. The method of FIG. 6 and/or one or moresteps thereof may be performed by a computing device (e.g., contentcontroller 125). The method illustrated in FIG. 6 and/or one or moresteps thereof may be embodied, for example, in computer-executableinstructions that are stored in a computer-readable medium, such as anon-transitory computer-readable memory. In some instances, one or moreof the steps of FIG. 6 may be performed in a different order and/orcombined. In some instances, one or more of the steps of FIG. 6 may beomitted and/or otherwise not performed. The results and data produced byFIG. 6 may be used in any other steps or processes disclosed herein, asneeded.

As seen in FIG. 6, the method may begin at step 601 in which the contentcontroller 125 may determine and/or otherwise receive group interestscores in one or more topics. The content controller 125 may thenperform one or more of steps 602-604. In step 602, the contentcontroller 125 may rank the topics of content items by popularity (e.g.,interest score) for each group with higher interest scores correlatingto higher popularity of the content item for the corresponding group. Instep 603, the content controller 125 may determine and/or otherwiseimplement customized content recommendations and/or experiences for agroup or for a user based on the group's interest scores and/or a userinterest scores. As an example, if one member of a group is highlyinterested in a particular topic, a content item having the particulartopic may be recommended to other members of the user's group byproviding the recommendation to the content presentation device tooutput to a member of the group. The member's response (positively ornegatively) may be recorded to update the group's interest score in theparticular content item. In step 604, the content controller 125 maygenerate reports including the interests scores (e.g., group interestscores, user interest scores), topic rankings determined in step 602,and customized content recommendations/responses determined in step 603,and may transmit the reports to one or more entities for use withnegotiation of the one or more advertisement slots.

FIG. 7 depicts an illustrative method of implementing one or moreaspects discussed herein. The method of FIG. 7 and/or one or more stepsthereof may be performed by a computing device (e.g., consumptionmonitoring system). The method illustrated in FIG. 7 and/or one or moresteps thereof may be embodied, for example, in computer-executableinstructions that are stored in a computer-readable medium, such as anon-transitory computer-readable memory. In some instances, one or moreof the steps of FIG. 7 may be performed in a different order and/orcombined. In some instances, one or more of the steps of FIG. 7 may beomitted and/or otherwise not performed.

As seen in FIG. 7, the method may begin at step 701 in which theconsumption monitoring system may determine a user profile. Theconsumption monitoring system may store multiple user profiles (e.g.,one for each family member in the household). The consumption monitoringsystem may, using one or more sensors discussed herein, identify theuser of the content presentation device. For example, the consumptionmonitoring system may determine which family member is using the contentpresentation device using facial recognition. Additionally oralternatively, the user may enter his or her identifier and/or otherwiselogin to his or her account and the consumption monitoring system mayuse this information to determine which family member is currently beingprovided the content items. The user profile may include the user'sinterest scores, records of past collected interest data and replacementcontent items, and the like.

In step 702, the consumption monitoring system may collect and/orotherwise receive user activity data and/or environmental data in asimilar manner as discussed above in step 301. Further, in step 703, theconsumption monitoring system may determine whether the user is in aconsumption state in a similar manner as discussed above in step 302. Ifthe user is in a consumption state, the consumption monitoring systemmay repeat steps 702 and 703. Otherwise, if the user is in anon-consumption state, the consumption monitoring system may generate anon-consumption state report (discussed above) and transmit thenon-consumption state report to the content controller 125 in step 704.In one example, the consumption monitoring system may select useractivity data and/or environmental data for the relevant timeframe forinclusion in the report in a similar manner as discussed above in step303. Further, the report may also include an indication of thenon-consumption state and may identify the user. At this point, thecontent controller 125 may receive the report and perform one or more ofthe steps of FIGS. 3 and 5.

FIG. 8 depicts another illustrative method of implementing one or moreaspects discussed herein. The method of FIG. 8 and/or one or more stepsthereof may be performed by a computing device (e.g., consumptionmonitoring system). The method illustrated in FIG. 8 and/or one or moresteps thereof may be embodied, for example, in computer-executableinstructions that are stored in a computer-readable medium, such as anon-transitory computer-readable memory. In some instances, one or moreof the steps of FIG. 8 may be performed in a different order. In someinstances, one or more of the steps of FIG. 8 may be omitted and/orotherwise not performed.

As seen in FIG. 8, the method may begin at step 801 in which theconsumption monitoring system may determine a user profile in a similarmanner as discussed above in step 701. In step 802, the consumptionmonitoring system may collect and/or otherwise receive user activitydata and/or environmental data in a similar as discussed above in step702.

However, rather than locally determining whether the user is consuming acontent item (e.g., whether the user is in a consumption state or anon-consumption state), the consumption monitoring system may, in step803, continuously or periodically send each (e.g., all) of the collecteduser activity data and/or environmental data to the content controller125 so that the content controller 125 may perform the determination ofwhether the user is consuming the content item and the relevanttimeframe. Offloading some or all of the consumption state processingmay allow the consumption monitoring system at the user premises to be athin device with less processing power, and thus potentially be a lessexpensive device to produce. This may further allow existing devices atthe user premises (e.g., STB, gateway, modem, etc.) to be reprogrammedto act as consumption monitoring systems, even though such existingdevices may not have sufficient resources to locally implement many ofthe steps in the various flow charts described herein. In someinstances, the consumption monitoring system may send a user identifierto the content controller 125.

While in the above examples, the content presentation device presentingthe content items was a television, the device may be any contentpresentation device, such as but not limited to a display device (e.g.,a computer monitor, tablet, smartphone, or any other display devicecapable of displaying video and/or audio content) or an audio-onlypresentation device (e.g., a streaming audio radio, automobile audiosystem, etc., capable of presenting audio content). The systems,apparatus, and methods described herein, for example, may be used toobtain and/or analyze user activity data and/or other information toascertain the user's environment, which may ultimately allow for adetermination of a reason why a user is not consuming (e.g., viewing)the content item.

The descriptions above are merely example embodiments of variousconcepts. They may be rearranged/divided/combined as desired, and one ormore components or steps may be added or removed without departing fromthe spirit of the present disclosure. The scope of this patent shouldonly be determined by the claims that follow.

1. A method comprising: providing a content item to a user.